Modernizing Medical Device V&V

In the medical device development industry, we have been doing something poorly, and worse still, we’ve been doing it poorly for well over 50 years (as far as I’m aware). This “thing” is functional verification and validation testing, or as it’s known in the jargon of our industry—V&V. Performing it has been an issue due to the extensive amount of time it takes to perform both activities.

Fortunately, in recent years, decent tools have come on the market to help us with the verification portion of it. These have enabled the automatic creation of mocks and stubs for the code under test. Tools that create mock stubs to support unit testing on the verification side have significantly accelerated test suite creation, allowing us to move into continuous integration and continuous delivery (CI/CD).

But unfortunately, that’s where the advances end. Validation testing, on the other hand, hasn’t seen much improvement, especially in speed. That fact is pretty amazing considering how much validation testing must cost: not only in the delayed introduction of this release candidate, but also just in having someone sit there and perform the testing for months.

I can remember many projects where it took more than six months to perform validation testing on a new release candidate. Multiple three-ring binders of test procedures were documented, and some poor employee (usually an engineer) had to stand there and manually enter commands to the unit under test for hours at a time. 

In addition, that’s only if the tests were successful. Obviously, if a test failed, either the test restarts (which is very frequent) with a new release candidate, or someone has to sit down and figure out what a regression test looks like. Sometimes that can take more time than just starting over.

Adding Cybersecurity

Cybersecurity has complicated this process even further. The main issue is that it requires a second full test program, stacked on top of the first, performed by a different group, with different tools, and is usually delayed until functional V&V has been completed.

Security also added the requirement for a response time for fielding patches and updates, which means V&V and cybersecurity testing all have to be automated and performed (and reperformed) quickly. Delays in fielding an update or patch for a vulnerability in a fielded medical device can have severe consequences, such as a mandatory Class 1 recall. We simply don’t have months anymore to perform validation testing. These have to be performed in a matter of hours.

Your quality system should cover both, and ISO 13485 spells out the steps. But the regulation doesn’t say testing has to be slow. The delay lives in how teams plan and run tests, and due to “that’s how we have always done it” (the worst justification you can have). Sometimes they throw in “liability” as well as justification for this slow process.

From product launches to M&A and manufacturing trends, the MPO Source delivers the medtech news you need twice a week. Subscribe here!

Automated Solutions

Manual testing is where schedules go to die. There are automated solutions for cyber-physical systems, such as medical devices.¹ Obviously, wherever possible, use regression analysis to determine what part of the test can remain without having to re-perform it. However, I have seen that process take days in some cases. 

If you have an automated validation testing system, such as the one being proposed in this article, regression analysis may look like the slow option. Unless it’s extremely obvious what can and can’t be reused in testing, it might just be easier to re-perform the entire validation test once it’s automated.

Validation automation also has the benefit of running the same tests the same way in the same order, every time. In addition, this testing consistency can extend to multiple abuse cases simultaneously, such as simulating user input at the same time as high communication traffic flows coming from a network connection. Being able to simulate these rare conditions consistently and repeatably is extremely useful for cybersecurity testing.

Cybersecurity Testing

According to the FDA, security testing follows the same quality logic as functional testing, and the agency points to the same ISO 13485 activities. However, cybersecurity adds tests that plain functional testing skips. The February 2026 premarket cybersecurity guidance in Section V.C “Cybersecurity testing” names four kinds of testing to be performed.

• Security requirements testing: Evidence that each security design input was implemented, plus a boundary analysis that defines what the manufacturer controls versus the surrounding systems, with the rationale for those boundary assumptions
• Threat mitigation testing: Evidence that risk controls actually work against the threats identified in the architecture views, covering security effectiveness, performance under maximum traffic, and stability
• Vulnerability testing: Scoped to ANSI/ISA 62443-4-1, this is the category with the most testing regimes: 
• User abuse cases
• Fuzz testing
• Attack surface analysis
• Vulnerability chaining
• SCA and SBOM generating/monitoring
• Closed-box scanning for known vulnerabilities
• Static (SAST) and dynamic (DAST) code analysis checks for insecure coding practices, including hardcoded, default, and easily guessed credentials
• Penetration testing: Independent testing that discovers and exploits vulnerabilities in the product, with a report that documents tester independence and expertise, scope, duration, methods, and findings

The FDA says security testing should run throughout the build, and early testing prevents the need to redesign or reengineer the device. This appears to be the agency endorsing the concept of shifting all cybersecurity testing left. Moving these from a manual end-of-project scramble to continuous execution is the difference between finding a vulnerable dependency the day it is coded or finding it during pre-submission testing. 

Compare the four kinds of testing against one question—“What can be automated?” Three of the four bend easily to machines; one does not (i.e., penetration testing). Penetration testing is the one type of testing that, at the moment, cannot be automated. That may not last forever, but that is the current state today. So, how do we speed up penetration testing today? 

What you need to do is micro-pen testing: scope a series of smaller pen tests on implemented subsystems of the overall device and perform these micro penetration tests during development. These individual systems get tested before the entire system is ready, before a release candidate is ready.

A number of interesting projects, some of which are open source and can be found online, are trying to address using large language models (LLMs) for penetration testing. So far, they are interesting curiosities rather than something rigorous enough for production use. But as with all things in the LLM environment, that can change overnight. 

Ongoing due diligence should focus on what new tools are coming online, rather than the advertising hype cycle you will see from lots of vendors. We need to focus on what can actually be done from the penetration testing side with an LLM before deciding to commit that fourth regimen of testing over into a fully automated system.

The same approaches can speed up both functional and cybersecurity V&V. Do the repeatable work continuously and automatically, and reserve human expertise for the work that needs judgment. On the functional side, that means automated regression and simulation. On the security side, it means the vulnerability-testing category running in the pipeline on every build, and penetration testing scoped tightly by a threat model.

Framed against FDA’s four testing categories, the plan almost writes itself. Three of the four are automatable and belong in continuous execution. The fourth, penetration testing, is made faster by feeding it better inputs rather than by replacing the tester. A development team that organizes its testing this way is not cutting quality or security corners to hit a date. It is spending its slowest, most expensive resource—expert human attention—only where that attention actually changes the outcome.

And who knows, given how fast LLM technology improves, maybe next month penetration testing can be performed by an LLM.

Reference
¹ www.adaptarobotics.com/robots/

More insights from this author: The New Minimum Elements Standard for SBOMs

Christopher Gates is the founder and CEO of arsMedSecurity, a medtech cybersecurity consulting firm. He is a recognized thought leader in medical device cybersecurity and the current co-chair for H-ISAC’s MDSC. Gates has more than 50 years of experience developing and securing medical devices and works with numerous industry-leading device manufacturers. He frequently collaborates with regulatory and standard bodies, including the CSIA, Health Sector Coordinating Council, H-ISAC, and Bluetooth SIG.